1. Field of the Invention
This present invention is directed to methods and apparatus for generating a modified intensity projection image from captured medical image data of a subject.
2. Description of the Prior Art
In the medical imaging field, several imaging schemes are known. For example PET (Positron Emission Tomography) is a method for imaging a subject in 3D using an injected radio-active substance which is processed in the body, typically resulting in an image indicating one or more biological functions.
In PET/CT, many important pathologies and anatomical structures appear as very high (or low) intensities. For example, a tumour in an FDG-PET image will often appear as a bright region.
A Maximum Intensity Projection (MIP) image is a useful way to visualise such medical images. Each pixel in a MIP is the maximal intensity along a ray orthogonal to the plane of the MIP. The resulting pixel values come from different depths along the rays and hence a MIP can be thought of as a simple form of 3D visualisation.
However, MIPs suffer from the following limitations:
1. A MIP provides little in the way of anatomical context and hence it is difficult to localise foci without examining the slice MPRs (multiplanar reconstructions). With FDG-PET, the physiological uptake does provide some anatomical context but is quite weak especially in areas such as the lung and mediastinum in which non-specific uptake is low.
2. An additional confusion arises from the ambiguity in orientation: it is often difficult to tell whether one is looking at the body from the anterior or the posterior.
3. The display can be difficult to interpret when multiple bright structures overlap. For example, when looking at a contrast-enhanced CT image of the thorax, the 3D nature of the thoracic structure obscures information of particular structures. In PET images, a large organ with high uptake such as the bladder, brain or heart can obscure all or most neighbouring hotspots. Similar problems can occur if two hot-spots happen to coincide for a particular viewing angle—the MIP must be rotated to separate the two hot-spots, something that can only be done with dynamic interaction of the MIP; it would not typically be useful to use such an image for a paper report.
In addition, fainter hotspots that are near to other hotter objects or even partially surrounded by them are difficult to see in the MIP. With reference to FIG. 1, one typical example of this would be a lung nodule (104) that is near to the left ventricle (102) in a PET scan. The left ventricle exhibits significant uptake and obscures a small nearby lesion (104), as shown in FIG. 1. This could also occur for lesions near the bladder, brain or a primary tumour.
FIG. 1 is a diagram of 3 frames (a, b, c) in a rotating MIP where the tumour 104 is obscured in some frames by high FDG uptake in the left ventricle, making it difficult to select
MIP displays are conventionally created by deriving a sequence of 2D views, each produced by casting rays from a slightly different angle around the body. By grouping these 2D views together and displaying them in a sequence as a cine, the MIP image appears to rotate. By rotating the angle of the MIP view (either manually or automatically), it is possible to differentiate many structures that are separated in space. For this reason, many medical imaging platforms will include the ability to change the angle of the MIP projection, and usually provide an animation of the MIP being rotated through 360 degrees.
Unfortunately, even when varying the angle of the MIP, some structures will remain occluded (e.g., if several structures lie near to each other and the axis of rotation). Also, this will not allow any visual separation of structures that are included within other structures (such as the hollow wind pipes within the trachea). It will also only allow visual separation of adjacent anatomical structures over a small range of angles, which may not be useful angles to view the anatomy.
Moreover, the MIP as created using the above process provides limited anatomical context for any regions of high activity that maybe of interest. For example, in lung cancer the location of any hotspots in the mediastinum region is important as this information may determine the prognosis and treatment options available to the patient.
In addition, to ensure that no lesions are missed a reading physician is often required to individually inspect every slice in an image to ensure that no lesions are missed, and the physician may not work off the MIP. Lesions that are missed during the slice-by-slice inspection will not be caught without a second slice-by-slice reading. Since the MIP obscures parts of the scan it cannot even be relied upon as a second check to ensure all lesions have been located.
Two previous systems, in U.S. Pat. No. 7,356,178 and U.S. Pat. No. 7,339,585, attempt to display information from relevant regions only, but this requires the regions to be selected in the first place. A system to fit an equation to the pixels lying along a projected line and displaying some representation of the equation has also been considered, in U.S. Pat. No. 7,218,765. This summarises all the information in the image, however the resulting images are unlikely to be readable without significant training and doesn't prevent multiple hotspots from being obscured. The MIP can also be used to change the values of the weights used for volume rendering, as in U.S. Pat. No. 7,250,949, but again the result shows a single 2D surface through the image which can obscure lesions. A method for displaying depth information with the MIP, as in U.S. Pat. No. 7,088,848, can indicate to the user that other lesions may exist, but a check of the individual slices is required to verify this.
The present invention aims to address these problems and provide improvements upon the known devices and methods.
Aspects and embodiments of the invention are set out in the accompanying claims.
In general terms, one embodiment of a first aspect of the invention can provide a method of generating from captured medical image data of a subject a modified intensity projection image for display, the method comprising: obtaining an intensity projection image data set from the image data; obtaining a secondary image data set from the image data, determining a region of interest in the secondary image data set; determining a surface of the region of interest; and combining the intensity projection image data set and the region of interest surface to generate the modified intensity projection image for display.
This allows an accurate combination of the information from two different image data sets, giving the clinician more context in analysing the intensity projection image.
Preferably the step of combining comprises combining the complete intensity projection image data set with only a portion of the secondary image data set, said portion delineated by the region of interest surface.
Suitably, the step of combining comprises blending the region of interest with the intensity projection image data set. This allows a simple means to combine the information from the image data sets.
More preferably, the step of combining comprises replacing part of the intensity projection image data set with the region of interest, wherein the part of the intensity projection image replaced by the region of interest is delineated by the region of interest surface. This provides a clear indication of the position of, for example, a lesion with respect to the region of interest.
Suitably, the step of combining comprises using the region of interest surface to determine a clipping point along an intensity projection ray.
In an embodiment, the step of combining comprises generating the modified intensity projection image from the intensity projection image data set using only those parts of the intensity projection image data set outside the region of interest surface.
Preferably, the secondary image data set is for a type of image different from the intensity projection image set.
Suitably, the image data comprises: the secondary image data set, captured using an anatomical imaging protocol; and the intensity projection image data set, captured using a functional imaging protocol. The anatomical imaging protocol may be, for example, a CT or MRI scan, and the functional imaging protocol may be, for example, a PET scan, such as FDG-PET.
Preferably, the method further comprises determining a registration between the intensity projection image data set and the secondary image data set.
More preferably, the surface of the region of interest is determined by a surface-rendering process.
One embodiment of a second aspect of the invention can provide a method of generating from captured medical image data of a subject a modified intensity projection image for display, the method comprising: obtaining an intensity projection image data set from the image data; obtaining a secondary image data set from the image data, determining a region of interest in the secondary image data set; and combining the intensity projection image data set and information from the region of interest to generate the modified intensity projection image for display, wherein the step of combining comprises combining the complete intensity projection image data set with only a portion of the secondary image data set, said portion contained by the region of interest.
One embodiment of a third aspect of the invention can provide a method of generating from captured medical image data of a subject a modified intensity projection image for display, the method comprising: obtaining an intensity projection image data set from the image data; obtaining a secondary image data set from the image data, determining a region of interest in the secondary image data set; and combining the intensity projection image data set and information from the region of interest to generate the modified intensity projection image for display, wherein the step of combining comprises replacing part of the intensity projection image data set with the part of the secondary image data set contained by the region of interest.
One embodiment of a fourth aspect of the invention can provide apparatus for generating from captured medical image data of a subject a modified intensity projection image for display, the apparatus comprising: a processor adapted to: obtain an intensity projection image data set from the image data; obtain a secondary image data set from the image data, determine a region of interest in the secondary image data set; determine a surface of the region of interest; and combine the intensity projection image data set and the region of interest surface to generate the modified intensity projection image for display; and a display device adapted to display the modified intensity projection image.
One embodiment of a fifth aspect of the invention can provide a method of generating, from medical image data of a subject captured by an imaging apparatus, a modified intensity projection image for display, the method comprising: obtaining, by a processor, an intensity projection image data set from the image data; obtaining, by a processor, a secondary image data set from the image data, determining, by a processor, a region of interest in the secondary image data set; determining, by a processor, a surface of the region of interest; combining, by a processor, the intensity projection image data set and the region of interest surface to generate the modified intensity projection image for display; and displaying the modified intensity projection image on a display device.
A further aspect of the invention can provide a media device storing computer program code adapted, when loaded into or run on a computer, to cause the computer to become apparatus, or to carry out a method, according to any preceding claim
The above aspects and embodiments may be combined to provide further aspects and embodiments of the invention.